A Modified Sigma-Pi-Sigma Neural Network with Adaptive Choice of Multinomials

Sigma-Pi-Sigma neural networks (SPSNNs) as a kind of high-order neural networks can provide more powerful mapping capability than the traditional feedforward neural networks (Sigma-Sigma neural networks). In the existing literature, in order to reduce the number of the Pi nodes in the Pi layer, a special multinomial Ps is used in SPSNNs. Each monomial in Ps is linear with respect to each particular variable σi when the other variables are taken as constants. Therefore, the monomials like σn i or σ n i σj with n > 1 are not included. This choice may be somehow intuitive, but is not necessarily the best. We propose in this paper a modified Sigma-Pi-Sigma neural network (MSPSNN) with an adaptive approach to find a better multinomial for a given problem. To elaborate, we start from a complete multinomial with a given order. Then we employ a regularization technique in the learning process for the given problem to reduce the number of monomials used in the multinomial, and end up with a new SPSNN involving the same number of monomials (= the number of nodes in the Pi-layer) as in Ps. Numerical experiments on some benchmark problems show that our MSPSNN behaves better than the traditional SPSNN with Ps.